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ABSTRACT 



The elements of statistical formulae that cause 



people to perceive them as difficult were studied, working from 
theoretical bases that involve cognitive load, motivational 
perception, and impasse driven learning. In a pilot study conducted 
to identify difficulties that novices perceive while learning 
statistical formulae, 22 students in an introductory statistics class 
participated as subjects. As novices, on the first day of class, 
students w«re asked to rate 60 formulae from statistics texts for 
confidence in their ability to learn to compute, apply, and 
understand each formula. Results suggest that something other than 
the difficulty of the concept influences student attitude. A protocol 
analysis was conducted for three adults with no training in 
statistics. Results indicate that confidence declines as the number 
of unfamiliar elements in a formula increases . Perceived difficulty 
increased as the amount of information to be processed increased, 
supporting the idea of cognitive overload. Five tables and two graphs 
present study findings. (Contains 13 references.) (SLD) 
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Identifying and Alleviating Problems Novices Have While Learning Formulae 

Introduction 

In this study I investigate what elements of statistical formulae cause people to 
perceive said formulae as difficult. The perception of difficulty is important because it 
affects how and what people study as well as the amount of time they devote to study. By 
understanding how people perceive formulae we can design our instruction to 
accommodate these perceptions. 

I approach this investigation from three major theoretical bases. The first 
theoretical base will concern cognitive load. Are students intimidated by certain formulae 
because the formulae contain to much information to be held in working memory? The 
second theoretical base will concern motivational perceptions. Are there aspects of certain 
formulae which cause students to perceive certain formulae as more difficult and thereby 
stimulate negative emotions that interfere with the efficient use of working memory. The 
third theoretical base involves impasse driven learning. When learners reach a cognitive 
impasse, are they able work around it. If learners can work around certain impasses and 
not others, how do the impasses they can work around and those they cannot work 
around differ? 

Cognitive Load 

A cording to current information processing theory (Anderson 1987, 1990), 
learning takes place in working memory. Working memory is similar to short term 
memory and might be described as conscious attention. While conscious attention is 
necessary for learning, it is not sufficient. For a new concept to be learned, i.e., stored in 
long term memory, it must be held in working memory long enough for it to have 



meaning. Since the capacity of working memory is limited, the amount of information that 
can be learned at any one tim** is likewise limited. 

Much has been written in expert-novice literature about the enhanced working 
memory capacity of experts (Chi, Glasser, Fair 1988,). This enhanced working memory 
appears to be more a function of experience than of general intelligence. As such, the 
enhanced working memory of experts is domain specific. DeGroot's (1966) classic study 
of differences between expert and novice Chess players illustrates these points. Through 
the use of protocol analysis deGroot analyzed in detail the depth that experts and novices 
searched the Chess board, the heuristics behind attack and defense strategies of both 
groups, and other aspects of the reasoning process. Although the experts made the right 
moves more often than the novices, there was nothing in the protocol analysis to indicate 
why. 

Next deGroot exposed players to a slide of the twentieth move of a Chest master's 
game. These were all games with which the players were unfamiliar. After five seconds the 
slide was removes and the players were asked to reproduce the positions of the Chess 
pieces on a Chess board. The Chess masters could reproduce nearly all of the pieces with 
little or no errors. The novice performance was much poorer (Posner 1988 p.xxx). 
However, when the experiment was repeated with the Chess pieces placed on the board in 
a random manner, the experts did no better than the novices. Furthermore, the experts 
were upset over the chaotic board positions (Anderson 1990 p. 280). 

Given the enhanced working memory capacity of experts, the obvious question is 
how do experts exceed the normal limitations of working memory? One primary strategy 
for overcoming the limitations of working memory is to link new information with old 
information. By linking bits of information together, the expert is cognitively able to move 
backward, forward, and laterally along chains of inference that are to long to be held in 
working memory. This cognitive strategy is known as "chunking." 
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Chunking requires a knowledge base with which new information can be linked. 
This has two implications for the educator. First, the educator must understand the 
cognitive background of the individual learner : Second, the educator must find ways to 
link new information to information the learners already posses. These implications 
present difficulties when educators attempt to introduce learners to an alien domain like 
statistics. Novices frequently cannot chunk the new information because it is so different 
they cannot link it to any prior knowledge. This means the novice must hold the entire 
inference chain of a new concept in working memory long enough for the new cc .ept to 
become meaningful. If the inference chain is short enough to be held in working memory, 
it can be learned by the novice: Otherwise, the novice may learn an incomplete or 
incorrect chain of inference. This problem is exacerbated if the novice is also required to 
remember the meaning of special symbols often used in a formal language like statistics. 
Most of the literature asserts that ultimately, working memory limitations are overcome by 
gradually acquiring domain specific inference rules. While this knowledge is necessary for 
the construction of adequate expository text, it is not sufficient. For the instructor, one 
primary requirement in knowing how different types of information impact working 
memory. 

Kosslyn's research (1980) reveled that working memory was necessary to 
transform a visual image to a mental representation. This implies that seeing something 
requires working memory. The visual factors that impact working memory are the amount 
of information contained in the visual image and the amount of space that the information 
occupied. While it took more time for Kosslyn's subjects to search information spread 
over a larger area, searching a large area required less working memory than identical 
information presented in a smaller space. 

Larkin and Simon (1987) study strengthened and then went beyond Kossyln's view 
of the working memory demands of a visual search. According to Larkin and Simon, the 
working memory demands of learning from a diagram were affected by 1) the amount of 
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information presented, 2) the area over which the information was spread, 3) the 
complexity of the information, and 4) .vhether the information was implicit or explicit. 

Information presented in a diagram was easier to access than information in an 
expository text. This meant that it took longer to search text than it did to search a 
diagram. Consequently, subjects learning from text had to hold information in working 
memory longer than those learning from diagrams. Furthermore, information about 
computations required more working memory than information about objects, e.g., more 
working memory was required to hold a/b in working memory than was required to hold 
abc in working memory. 

Where Larkin and Simon used physics problems for their study, Sweller, Chandler, 
Tierney, and Cooper (1990) conducted similar research using coordinate geometry 
problems. For this research three groups were used. The first group received instruction in 
the from of a conventional expository test. The second group received a conventional 
worked out example. The conventional worked out example had a diagram at the top of 
the page and an explanation at the bottom. The third group received a modified worked 
out example. The modified worked out example had the text written on the diagram next 
to what the text explained. 

Subjects in the third group (modified worked out example) finished the problem 
quicker and made fewer problems than subjects in the other two groups. Subjects in the 
first group (conventional expository text) took longer and made more mistakes than 
subjects in the other two groups. The conclusion was that the length of time a subject had 
to hold information in working memory while searching for other relevant information had 
a negative impact on performance. 

Fear and Loathing 

^The previous section of this paper assumes that learners take a reasoned rational 
approach to learning statistical formulae. The assumption is that learners will understand 
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the information they are given, be able to locate relevant information, process the 
information, chunk the information, and move on. This may be an incorrect assumption for 
some learners. If a learner cannot understand the information presented, locate relevant 
information, process the information, or chunk the information, then what happens? 
According to Mandler(1989), 

" ...on a majority of occasions, visceral arousal follows the occurrence of 
some perceptual or cognitive discrepancy or the interruption or blocking of 
some ongoing action. Such discrepancies and interruptions depend to a 
large extent on the organization of mental representation of thought and 
action.... these discrepancies occur when the expectations of some schema 
are violated (p. 8)." 

Mandler goes on to say that the arousal caused by the blockage pre-empts other 
information held in working memory. The reduced working memory capacity prevents the 
full utilization of the cognitive apparatus. The thought process may become simplified 
(stereotyped and canalized) thus causing the learner to revert to simpler modes of problem 
solving (p. 9). 

This visceral reaction is intensified when people feel they should know something 
but do not (Weiner 1993). Anyone who reaches graduate school has had at least two 
courses in Algebra (high school and college). They feel that they should understand 
formulae. When they encounter a formula they do not understand they feel a sense of 
personal failure. To use Weiner's terminology, they feel they have sinned. These intense 
negative emotions waste enormous amounts of working memory. This inefficient use of 
working memory inhibits the ability to concentrate which intensifies the feelings of failure. 
Eventually the learner is overcome by frustration and guilt. If these reactions are intense 
enough or frequent enough, they can become a conditioned response, i.e. learned 
helplessness. In this case the conditioning must be overcome before learning can occur. 

Ideally, the educator will not present the learner with more information than the 
learner can hold in working memory or assume prior knowledge the students do not have. 
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In fact, this frequently happens (Reif & Larkin 1991, Larkin & Raniard 1984). Experts 
(teachers) often assume prior knowledge on the part of the novices (learners) which does 
not exist. The teacher's assumption that students should have the relevant prior knowledge 
gets passed to the students. If the students accept this assumption then one could expect 
them to feel that this lack of knowledge is due to a flawed sinful nature on their part. The 
feeling of having sinned causes feelings of shame and guilt. The feelings of shame and guilt 
prevent the students from asking for help because asking for help would require an 
admission of their sinful nature. Since no one asks for help, the teacher assumes that he or 
she is teaching and the students are learning. The truth is not known until it is time for an 
evaluation. Once the evaluation reveals the sinful nature of the class, feelings of shame and 
guilt get replaced with feelings of anger and hostility. If the teacher pontificates, these 
feelings of anger and hostility are exacerbated. Once a class reaches this point, it is 
difficult for the teacher to teach or the learner to learn. 

Impasse Learning 

In contrast to the previous section of this paper, VanLehn's concept of impasse 
driven learning (1990) postulates that when a student encounters the blockage of some 
ongoing cognitive activity (an impasse), the learner will attempt to work around the 
blockage. To use information processing terminology, when a person can no longer use an 
algorithm to solve a problem, he or she reverts to the use of weak methods, i.e. heuristics, 

to work around the blockage. 

On encountering a new situation, people attempt to match the situation to similar 
patterns stored in memory. These patterns are called fetch patterns. Fetch patterns fetch 
facts and rules from declarative knowledge. The rules they fetch match "the most specific 
patterns consistent with positive training instances (p. 152)." Fetch patterns tend to be 
overly specific, i.e. they do not lend themselves to generalizations. If applicable fetch 
patterns cannot be located then o ie is at an impasse. 
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Once rules have been fetched, they are used as predicates for deciding which rule 
to execute. VanLehn refers to these rules as test patterns (p. 89), "Test patterns are the 
maximally specific patterns consistent with the examples (p. 154)". There are two 
situations where test patterns can cause an impasse. As with fetch patterns, overly specific 
test patterns cause an impasse. Unlike fetch patterns, test patterns can also be overly 
general. An overly general test pattern cannot discriminate well enough between relevant 
facts and rules to lead to a conclusion. If no conclusion can be reached then one is at an 
impasse. 

As mentioned previously, when an impasse is encountered the learner reverts to 
heuristics in order to work around the impasse. If an solution is found, whether correct or 
incorrect, then the solution is stored in memory as a rule. Using computer terminology, 
VanLehn calls the incorrect procedures "bugs." Once bugs are stored in memory they 
become rules which are applied to other similar situations. In this manner the bugs spawn 
more bugs. Buggy procedures are easy to correct once they are identified. Once one is 
aware of the existence of bugs they are easy to identify, locate, and correct. The ultimate 
solution for the educator is to identify potential bugs and design instruction to eliminate 
them before they manifest. 

VanLehn never discusses the visceral reactions to cognitive blockage mentioned by 
Mandler. Apparently motivation is not an issue with impasse driven learning. When 
cognitive activity is blocked, the student simply works around the blockage. 

Pilot Study 

A pilot study was conducted to identify difficulties that novices perceive while 
learning statistical formulae. If the perceived difficulties are due to having to hold to much 
information in working memory at one time then longer formulae should be perceived as 
more difficult that shorter formulae. Furthermore, formulae which required learners to 
hold computations in working memory should be perceived as more difficult than formulae 
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which do not require such an activity. Formulae that present a more difficult concept 
should also be rated as more difficult than those presenting a simpler concept. If the 
perceived difficulty is due to visceral reactions caused by cognitive blockages which result 
from lack of adequate information, then formulae with more of these elements should be 
perceived as more difficult. If subjects can successfully work around any cognitive impasse 
encountered in a formulae, then there should be little difference in the perceived difficulty 
of said formulae. If subjects cannot successfully work around impasses, can factors that 
prevent this be identified? 

Method - 

Subjects 

The subjects were twenty two students in and introductory statistics class. The 
data was collected before the first day of class. 

Procedures 

Subjects were given sixty formulae taken from introductory statistics books. The 
formulae covered the mean, the median, variance, standard deviation, regression and 
correlation. Subjects used a seven point Likert scale to rate their confidence in their ability 
to learn to compute, apply, and understand each formula. A score of one on the Likert 
scale meant "not certain" a score of seven meant "absolutely certain." 

Results 

After collecting the data, the mean scores for each aspect (compute, understand, 
apply) of each formula were correlated. The correlations were all over .95. Apparently the 
subjects saw little difference between learning to compute apply and understand the 
formulae. For this reason, the rest of the analysis was only done on the mean scores for 
compute. 

10 



APPLY 
UNDERSTAND 



Compute Apply 
0.963 

0.969 0.981 



Table One. Pearson correlation matrix for mean scores on the students confidence in 
their ability to learn to compute, understand and apply statistical formulae. 

Next, the mean scores were used to rank the formulae. A ttest between the highest 
and fifth highest scores (absolutely certain) gave a non-significant p value. A ttest 
between the lowest and fifth lowest (not certain) scores gave a p.<.001. 

Discussion 

While there was no significant difference in the scores of the lower ranked (easier) 
formulae, there were significant differences in the scores of the higher ranked (harder) 
formulae. One might assume that formulae presenting more difficult concepts would be 
considered more difficult to learn than those presenting easier concepts. To a certain 
extent this was true. Formulae for correlation were ranked higher (more difficult) than 
formulae for the mean. However, one formulae for the median was ranked in the lowest 5 
formulae (least difficult) while two formulae for the median were ranked in the highest five 
formulae (most difficult). Clearly there was something beyond the difficulty of the concept 
causing the student to perceive the formulae as difficult. 

Protocol Analysis 

Method 

In order to gain a greater focus on the factors causing the students to perceive 
certain formulae as difficult to learn to learn, a protocol analysis was conducted. 
Subjects 

The subjects for the protocol analysis were not in college at the time. The 
first subject was a grade school teacher. The second subject was an Attorney, the third 



ii 



subject was a middle school teacher with an Masters degree in education. None of them 

had any training in the use of statistics. 

Procedure 

Like the class, the subjects in the protocol analysis used a seven point Likort scale 
to rate their confidence in their ability to learn to compute each formulae. In addition these 
subjects were asked to identify and count the number of things in each formula that mead 
the formula difficult. The first subject in the protocol analysis identified four variables that 
she thought might make the formulae difficult to learn. These variables were, 1) unfamiliar 
objects (Greek letters, subscript, capital letters, and words), 2) familiar objects (numbers 
and lower case English letters), 3) uncommon mathematical operators (square root, 
brackets), and 4) common mathematical operators (addition, multiplication, subtraction, 
division, and squaring). As she worked through each formula she counted and recorded 
the number of variables in each category of each formula. As she was finishing with the 
formi f ae, she realized that she had not identified the inferred multipliers (bx). Rather than 

start over, a fifth category for inferred multipliers was added. Subjects two and three 

added no new categories of variables and included the inferred multipliers in category of 

common mathematical operators. 

In order to identify the subject's score most representative of the class, the scores 

of the subjects were correlated with the mean scores of the class. The results are shown in 

the following table. 

PEARSON CORRELATION MATRIX 

CLASS MEANS SUBJECT 1 SUBJECT 2 SUBJECT 3 

SUBJECT ONE 0.737 1.000 

SUBJECT TWO -0.076 0.116 1-000 

SUBJECT THREE 0.506 0.655 0.060 1-000 

Table 2. Correlation matrix for protocol analysis subjects and class means. 



ERIC 



12 



Next the scores of the subjects were plotted against the class mean scores and n 
lowess line was added. The results are in graph 1 . 

Discussion 

None of the subjects in the protocol analysis seemed to be affected by the length of 
a formula. They seemed to chunk the relevant information and move on. The problems 
occurred when they encountered information which they did not understand. On several 
occasions subject one said, "I can't learn to do this. I don't want to learn to do this." 

Subject three reacted even more violently. She repeatedly apologized for "not 
being good at math." Several times she covered a formula with her hands and physically 
recoiled from it. Before we finished, her reactions became too intense to continue. She 
asked if she could take the formulae home and return them later. It was several weeks 
before she finished ranking the formulae and categorizing the elements in each formula. 
She indicated that she considered the task extremely difficult. 

By contrast, subject two took a reasoned rational approach to ranking the 
formulae and categoi izing the elements. When encountering something that he did not 
understand subject two would say, "If you tell me what that means then I can work this 
formula." 

The protocol also reveled the reason two formulae for the median were perceived 
as difficult. When encountering a formulae likeBS. The subjects asked questions like, "Is 
that M times ed or M times e times d, or is Med on thing?" Subject three was upset over 
the inclusion of words in some of the formulae. Subject one considered Med, L, cum, and 
yhied unfamiliar objects. All subjects in the protocol analysis were confused by the 
presence of uppercase letters in formulae. Greek letters distressed subjects one and three. 
Subject three recognized the upper case Sigma used in many of the formulae but 
repeatedly apologized for not knowing what it meant. 
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a - . T {S&N-cum f) 

Med = L + - — - — — 

/med 



Formula B5. 



Analysis of Pilot Data Using Information Gained From the Protocol Analysis 



Since subject one appeared to be most representative of the class, her counts of the 
unfamiliar objects, familiar objects, uncommon mathematical operators, common 
mathematical operators, and inferred multipliers were used as independent variables in a 
multiple regression analysis. The class mean scores were used as the dependent variable. 
The results are shown in the following table. 



DEPVAR: MEAN SCORES COMPUTE N: 60 MULTIPLE R: 0.723 SQUARED MULTIPLE R: 
0.523 

ADJUSTED SQUARED MULTIPLE R: .479 STANDARD ERROR OF ESTIMATE: 0.331 

VARIABLE COEFFICIENt STD STD TOLERANCE T P(2 TAIL) 

ERROR COEF 



CONSTANT 


4.910 


0.110 


0.000 




44.662 


0.000 


COMMON OBJECTS 


-0.004 


0.026 


-0.022 


0.455 


-0.159 


0.874 


COMMON OPERATORS 


-0.046 


0.025 


-0.271 


0.413 


-1.855 


0.069 


UNFAMILIAR OPERATORS 


0.062 


0.040 


0.228 


0.423 


1.575 


0.121 


UNFAMILIAR OBJECTS 


-0.050 


C.023 


-0.360 


0.322 


-2.175 


0.034 


INFERRED MULTIPLIERS 


-0.112 


0.039 


-0.457 


0.344 


-2.854 


0.006 


ANALYSIS OF VARIANCE 










SOURCE SUM-OF-SQUARES DF 


MEAN-SQUARE 


F-RATIO 


P 






REGRESSION 6.497 


5 


1.299 


11.851 


0.000 






RESIDUAL 5.921 


54 


0.110 











Table 3. Multiple Regression with subject one's count of common, objects common 
operators, unfamiliar objects, and inferred multipliers as independent variables. 
The dependent variable is the class mean score for its ability to learn to compute the 
formulae. 



Using some exploratory data analysis techniques, the number of unfamiliar objects, 



familiar objects, and common mathematical operators were reduced to the .3 power. 
Using this transformed data increased the squared multiple r from .523 to .721. 

DEPVAR: MEAN SCORES COMPUTE N: 60 MULTIPLE R: 0.849 SQUARED MULTIPLE R: 
0.721 

ADJUSTED SQUARED MULTIPLE R: .695 STANDARD ERROR OF ESTIMATE: 0.253 



VARIABLE COEFFICIENT STD STD TOLERANCE T P(2TAIL) 

ERROR COEF 



CONSTANT 


6.127 


0.265 


0.000 




23.090 


0.000 


COMMON OBJECTS 


-0.473 


0.159 


-0.282 


0.571 


-2.970 


0.004 


COMMON OPERATORS 


0.000 


o.ooc 


0.107 


0.568 


1.119 


0.268 


UNFAMILIAR OPERATORS 


0.067 


0.029 


0.245 


0.449 


2.287 


0.026 


UNFAMILIAR OBJECTS 


-0.860 


0.108 


-0.842 


0.459 


-7.934 


0.000 


INFERRED MULTIPLIERS 


-0.032 


0.026 


-0.130 


0.464 


-1.234 


0.222 



ANALYSIS OF VARIANCE 

SOURCE SUM-OF-SQUARES DF MEAN-SQUARE F-RATIO P 

REGRESSION 8.953 5 1.791 27.906 0.000 

RESIDUAL 3.465 54 0.064 

Table 4. Multiple Regression with transformed data. 



Stepwise regression (table 5) with the transformed data eliminated the common 
mathematical operators and reduced the squared multiple squared r to 0.715. 



DEP VAR: MEAN SCORES COMPUTE N: 60 MULTIPLE R: 0.845 SQUARED MULTIPLE R: 
0.715 

ADJUSTED SQUARED MULTIPLE R: .694 STANDARD ERROR OF ESTIMATE: 0.254 



VARIABLE COEFFICIENT 


STD STD 


TOLERANCE 


T P(2 TAIL) 






ERROR COEF 








CONSTANT 


6.026 


0 250 


0.000 




24.079 


0.000 


UNFAMILIAR OBJECTS 


-0.825 


0.104 


-0.807 


0.502 


-7.934 


0.000 


COMMON OBJECTS 


-0.416 


0.151 


-0.248 


0.637 


-2.751 


0.008 


UNFAMILIAR OPERATORS 


0.079 


0.028 


0.287 


0.511 


2.848 


0.006 


INFERRED MULTIPLIERS 


-0.045 


0.023 


-0.184 


0.584 


-1.951 


0.056 



ANALYSIS OF VARIANCE 

SOURCE SUM-OF-SQUARES DF MEAN-SQUARE F-RATIO P 

REGRESSION 8.873 4 2.218 34.413 0.000 

RESIDUAL 3.545 55 0.064 

Table 5. Stepwise Regression with transformed data. 



Discussion 

The results of the multiple regression show that as the number of unfamiliar 
objects increased by P the class's confidence in its ability to learn to compute a formula 
decreased by .86. Using stepwise regression the coefficient changed to -.825 Since the 
students' confidence in their ability to learn to compute a formulae was based on a seven 
point scale, these results indicate that as the number of unfamiliar objects increase beyond 
one, the students' confidence in their ability to learn to compute a formula should decrease 
dramatically. In graph 2 a scatter plot of the transformed unfamiliar objects plotted against 
the class mean scores illustrates this. E-en though the change was less dramatic, the 
unfamiliar mathematiwl operators also contributed to the model. Many formulae 
contained no unfamiliar mathematical operators and those that had them did not contain 
very many. For this reason the significance of the unfamiliar mathematical operators may 
be under estimated. In any case, the significance of the unfamiliar objects and the 
unfamiliar mathematical operators is consistent with the visceral reaction that Mandler 
predicts. The further contribution to the model made by the common objects and the 
inferred multipliers is consistent with the concept of cognitive overload. Specifically, the 
perceived difficulty of a task increases as the amount of information to be processed 
increases. It is surprising that the common mathematical operators did not contribute to 
the model. It is possible that the subjects had over learned the concepts behind the use of 
the common mathematical operators and were thus able to chunk this information and 
move on. However, it seems the same would be true of the inferred multipliers. The 
marginal contribution of the inferred multipliers may be explained by the confusion that 
arose when the subjects attempted to determine whither a string of letters was a word, an 
abbreviation, or several unknowns. 
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General Discussion 

Even though the subjects were instructed to rate their confidence in their ability to 
learn to compute the formulae, the still perceived formulae with more unfamiliar objects 
as more difficult than those with less. Furthermore, small increases in the number of 
unfamiliar objects in a formula caused dramatic decreases in the subjects confidence in 
their ability to learn to compute the formula. From the protocol analysis it seems that 
subjects rapidly reach a point where further information processing, and therefore further 
learning ceases. The implies that instructors and texts should 1) when possible, avoid 
formulae with unnecessary unfamiliar objects in them, 2) not introduce more than one 
unfamiliar object at time, and 3) introduce any unfamiliar objects prior to their use. This 
may seem obvious to some, but many instructors and statistics texts introduce formulae, 
complete with unfamiliar symbols, and then give the explanation. This requires the learner 
to hold decontextualized information as well as all possible links to the information in 
working memory and still process the information. If the symbols are introduced prior to 
their use, then that information can be proceduralized. Since the use of procedural 
knowledge requires little working memory, more working memory will available for 
processing information, i.e., understanding the formula. 
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Graph 1. Subject scores plotted against class scores 
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Graph 2. Transformed unfamiliar objects plotted against class scores 




